The HSM image is really just to run some e2e hsm tests. It's not being distributed and should not be used.

Yeah, e2e tests are now failing. Probably just revert the changes here

https://github.com/ory/hydra/actions/runs/12523819546/job/34933781514?pr=3914

/var/lib/softhsm/tokens is created when installing the softhsm / softhsm2 package. Ownership is root:softhsm. As is the token dir/files generated by the earlier pkcs11-tool command.

Rather than chown -R this directory and contents, it'd be wiser to just add ory to the softhsm group. However, the token generated only allows access to the file owner (root):

$ ls -l /var/lib/softhsm
drwxrws--- 2 root softhsm 4096 Apr  1  2024 tokens

$ ls -l /var/lib/softhsm/tokens
drwx--S--- 2 root softhsm 4096 Jan  5 22:34 c0e04acd-3e15-0266-6003-f9394edce34b

$ ls -l /var/lib/softhsm/tokens/c0e04acd-3e15-0266-6003-f9394edce34b
-rw------- 1 root softhsm   8 Jan  5 22:34 generation
-rw------- 1 root softhsm   0 Jan  5 22:34 token.lock
-rw------- 1 root softhsm 320 Jan  5 22:34 token.object

So in this case chown is required for the owner specifically. Probably shouldn't mess with the group ownership? 🤷‍♂️

Personally unless there's a clear reason for building an image to run with a non-root user, I'd suggest keeping it simple to maintain and just using root which avoids all this.

Anyone bind mounting a volume for local storage to the container is not going to have the ory UID/GID match their host user since regardless of the previous implicit UID/GID (100:101) or the static 500:500 proposed here, the typical non-root host user is 1000:1000.

That leaves the remaining benefit as a "best practice", but AFAIK is mostly moot if the user instead runs the container with Podman/Docker in rootless mode which uses /etc/subuid + /etc/subgid tables on the host to map the containers root UID + GID to the hosts user and any other container UID/GID values to the subuid / subgid range. Alternatively, when run as rootful, the container can be run with all capabilities dropped, which is implicit when running a process as a non-root user in the container.

Personally unless there's a clear reason for building an image to run with a non-root user, I'd suggest keeping it simple to maintain and just using root which avoids all this.

That leaves the remaining benefit as a "best practice", but AFAIK is mostly moot if the user instead runs the container with Podman/Docker in rootless mode which uses /etc/subuid + /etc/subgid tables on the host to map the containers root UID + GID to the hosts user and any other container UID/GID values to the subuid / subgid range. Alternatively, when run as rootful, the container can be run with all capabilities dropped, which is implicit when running a process as a non-root user in the container.

[...] mostly moot if the user instead [...]

security is never moot if you assume that everyone is smart or experienced enough to do the right thing. Running as root will fail several CI checks, please revert the user changes made

Uhh, I generally don't see people understanding non-root practice in containers, just pushing for it as a security practice regardless of knowing why.

As I've said, you'll have default capabilities granted to the container root user, but these aren't the same as host root (full caps). Switching to a non-root user just drops these additional caps, which if you really need to you can do so yourself as a security practice explicitly (dropping all caps, or when compatible, switching the user to non-root if you really insist).

The other reason to use non-root is if container escape is accomplished, then escaping as the root user to be root on the host would allow more damage, but this doesn't make non-root in a container exempt from this, you can still escape as non-root and become root on the host depending on the vulnerability exploited. The bulk of vulnerabilities require granting capabilities or relaxing security intentionally for a container than would otherwise be permitted by default, hence why I'm not really sold on the non-root container user "best practice".

I've seen a variety of containers that adopted non-root as a security practice, but then did workarounds that made security worse. Some granted the non-root user access to the docker socket, others granted capabilities that wouldn't be allowed by default, or mandated capabilities to support non-root but in such a way that prevented someone who is not using a feature that requires that capability and knows how to properly lock down a container to be forced to grant the capability for the binary to run...

To assume that building a container that runs as non-root is always more secure would be a mistake. Inexperienced users that make a container insecure can happen regardless too, trying to address incompetence with a blanket "fix" which often introduces more caveats, especially when the image fails to document this practice is ill-advised IMO. If you're going to defend the practice, have a good reason beyond parroting / hand-waving the justification.

More security conscious (even those inexperienced) should be using rootless Docker/Podman, where it's perfectly ok to have the container user as root, since escaping the container does not result in retaining the same UID/GID of the container user thanks to user namespace mapping.

Root in a container is not equivalent to root on the host. I've explained and demonstrated this so many times in the past to those that don't know better but parrot paranoia insisting that they know what they're talking about.

Running as root will fail several CI checks, please revert the user changes made

I'm not sure what CI checks you were referring to, but your response is to a comment that proposed dropping the non-root approach. The PR respected the non-root decision, all it did here was ensure consistency with the other images for non-root user setup, thus nothing to revert.

Proposal to consider removing this in future follow-up PR.

• Only seems relevant as a common "best practice" that some users request. It has it's fair share of caveats though, especially when not clearly communicated/documented for the image.
• The only known dependency on this is optional SQLite for a fixed DSN location at /var/lib/sqlite, but this becomes redundant if the container has a bind mount volume, which again is not documented for clarity, but a common practice for persistence vs named data volumes.
• Without any other support needed in the container, a user that wants this could better run the container with the user option to set the user to switch to and gain the same benefits.
• If Hydra ever needs to use some linux capabilities, you'd need to add a step with setcap to grant them for non-root users, and ideally at runtime in Go handle checking this to raise the capability rather than enforce it via setcap (especially if the feature using it is optional). With root (including rootless Docker/Podman), the need for setcap is avoided. Keep this simple to maintain, since historically in this repo the Docker support is already showing inconsistencies in maintenance, mostly due to redundant noise for running as a non-root user by default.

I've added commentary here, but from what I understand this path is only relevant when it's part of the DSN configured like in quickstart.yml? While ownership is only adjusted to support the non-root ory user to have RW access.

Some platforms like OpenShift have a feature that uses the root group for access while randomizing the UID the container runs as. I'm not entirely sure how viable that approach is and raised some concerns at another repo (caddy-docker) where someone proposed supporting that.

However, group/other permissions need to be adjusted for the DSN path regardless so that the hydra process has write access to create the DB (the execute bit on the directory is also required for that operation), thus 770 (rwxrwx---) is necessary.

services:
  hydra:
    image: localhost/hydra:v2.2.0
    # No delay in shutdown (uses tini as PID 1), proper signal forwarding + reaping:
    init: true
    # Optional: Run as non-root user, but use the root group for /var/lib/sqlite to avoid needing a chown
    user: 500:0
    environment:
      # DSN to SQLite:
      # https://www.ory.sh/docs/self-hosted/deployment#sql-persistent
      DSN: sqlite:///var/lib/sqlite/db.sqlite?_fk=true
    # Unlike the current quickstart.yml example which relies upon the Docker Compose `depends_on`
    # feature to apply DB migrations through another container instance, just run both commands here:
    # NOTE: While the quickstart doesn't mention it, `migrate sql` should technically
    # have you manually create a backup copy prior to running it (take caution with restart policy if automating):
    # https://www.ory.sh/docs/hydra/self-hosted/dependencies-environment
    entrypoint: ["ash", "-c"]
    command:
      - |
        hydra --config /etc/hydra/config.yml migrate sql --read-from-env --yes
        hydra --config /etc/hydra/config.yml serve all --dev --sqa-opt-out
    # Build image locally (no need to try pull from a remote registry):
    pull_policy: build
    build:
      dockerfile_inline: |
        FROM alpine
        RUN install --mode 770 --directory /var/lib/sqlite
        COPY --from=oryd/hydra:v2.2.0 /usr/bin/hydra /usr/bin/hydra
        # `--chmod` only required when container user is not run as root,
        # Volume mounting the config instead is fine as `git clone` permissions use umask which should result in 644.
        ADD --chmod=640 https://raw.githubusercontent.com/ory/hydra/refs/heads/master/contrib/quickstart/5-min/hydra.yml /etc/hydra/config.yml

So that works well, and if the user bind mounts a host directory instead, they can just set the user field to match UID of their host user. This avoids needing to set permissions of /var/lib/sqlite to 770, or know the UID/GID for the containers ory user/group is so that they can chmod their host directory ownership to that for compatibility 👍 (both inconvenient)

Nice! The dockerfile_inline would probably need to build the binary inline as well, otherwise the binary type might mismatch (osx versus linux for example)

otherwise the binary type might mismatch (osx versus linux for example)

I don't have macOS environment to verify, but the compose example above is just for a quick example. It copies the binary from the platform image it has, which AFAIK there is no macOS platform for registries/containers, they only publish windows and linux images?

Unless you were referring to architecture like AMD64 vs ARM64? That shouldn't be an issue, the directive should be using the inferred platform for selecting the image to copy from. Equivalent to FROM oryd/hydra or docker pull oryd/hydra without --platform= specified for either to change the default.

The example was focused on showing how much simpler the image is without catering to a non-root user, allowing the deployment to explicitly opt-in to non-root by choosing their own UID/GID, instead of needing to be mindful of what was chosen internally.

That's possible since the image doesn't have any real dependency lock-in on a non-root user being configured. They could just as easily take the current non-root images and run them as root in the same manner.

Context is lacking a bit for these COPY instructions.

Since the image itself doesn't have a build stage, it's rather vague what the hydra binary was built with. I doubt it matters for most, but since this is used for the official image published to DockerHub, some context might be worthwhile.

I assume going forward both Dockerfile-alpine + Dockerfile-distroless will take the binary built via Dockerfile-build, but perhaps you typically build Hydra outside of a container? 🤷‍♂️

Outside of docker compose and make docker, the binaries are built by goreleaser and then injected into the containers. So this is correct here.

As covered by earlier review comments, this could be dropped in a follow-up PR.

Technically a breaking change (but so is the 500:500 UID/GID proposed in this PR), but all a user should need to do is adjust the ownership of the /var/lib/sqlite or run with --user 101:101 (might differ by image depending on implicit UID/GID assigned to ory).

I don't agree with this change, the hydra command should run as non-root in my view

There is no suggestion snippet to remove the line. It was simply shifted to the end of the file. A change to drop it would be:

My comment only proposed dropping non-root, and noted that this PR configures the non-root user with a stable UID/GID value, rather than relying on it implicitly (unstable as any package installed prior may affect user/groups causing the UID/GID for ory to change unexpectedly).

I then noted that the stable UID/GID change itself, would be a breaking change to existing users of the image going forward, similar to if changing from non-root to root.

Since there's no real dependency on the non-root user in the image, it's fine and someone can choose to explicitly run as root at runtime (which should be secure, especially via rootless container).

This file is redundant AFAIK, I don't know where it's being used/published, so proposed changes for consistency remain here and the file could be dropped in a follow-up PR, reverting the change if required.

I checked and we no longer distribute this scratch image but instead the distroless variant. It should be OK to be removed.

This file is redundant AFAIK, I don't know where it's being used/published, so proposed changes for consistency remain here and the file could be dropped in a follow-up PR, reverting the change if required.

Let's remove it, I also could not find any use for it.

Uhh, I generally don't see people understanding non-root practice in containers, just pushing for it as a security practice regardless of knowing why.

As I've said, you'll have default capabilities granted to the container root user, but these aren't the same as host root (full caps). Switching to a non-root user just drops these additional caps, which if you really need to you can do so yourself as a security practice explicitly (dropping all caps, or when compatible, switching the user to non-root if you really insist).

The other reason to use non-root is if container escape is accomplished, then escaping as the root user to be root on the host would allow more damage, but this doesn't make non-root in a container exempt from this, you can still escape as non-root and become root on the host depending on the vulnerability exploited. The bulk of vulnerabilities require granting capabilities or relaxing security intentionally for a container than would otherwise be permitted by default, hence why I'm not really sold on the non-root container user "best practice".

I've seen a variety of containers that adopted non-root as a security practice, but then did workarounds that made security worse. Some granted the non-root user access to the docker socket, others granted capabilities that wouldn't be allowed by default, or mandated capabilities to support non-root but in such a way that prevented someone who is not using a feature that requires that capability and knows how to properly lock down a container to be forced to grant the capability for the binary to run...

To assume that building a container that runs as non-root is always more secure would be a mistake. Inexperienced users that make a container insecure can happen regardless too, trying to address incompetence with a blanket "fix" which often introduces more caveats, especially when the image fails to document this practice is ill-advised IMO. If you're going to defend the practice, have a good reason beyond parroting / hand-waving the justification.

More security conscious (even those inexperienced) should be using rootless Docker/Podman, where it's perfectly ok to have the container user as root, since escaping the container does not result in retaining the same UID/GID of the container user thanks to user namespace mapping.

Root in a container is not equivalent to root on the host. I've explained and demonstrated this so many times in the past to those that don't know better but parrot paranoia insisting that they know what they're talking about.

Running as root will fail several CI checks, please revert the user changes made

I'm not sure what CI checks you were referring to, but your response is to a comment that proposed dropping the non-root approach. The PR respected the non-root decision, all it did here was ensure consistency with the other images for non-root user setup, thus nothing to revert.

There is no suggestion snippet to remove the line. It was simply shifted to the end of the file. A change to drop it would be:

My comment only proposed dropping non-root, and noted that this PR configures the non-root user with a stable UID/GID value, rather than relying on it implicitly (unstable as any package installed prior may affect user/groups causing the UID/GID for ory to change unexpectedly).

I then noted that the stable UID/GID change itself, would be a breaking change to existing users of the image going forward, similar to if changing from non-root to root.

Since there's no real dependency on the non-root user in the image, it's fine and someone can choose to explicitly run as root at runtime (which should be secure, especially via rootless container).

otherwise the binary type might mismatch (osx versus linux for example)

I don't have macOS environment to verify, but the compose example above is just for a quick example. It copies the binary from the platform image it has, which AFAIK there is no macOS platform for registries/containers, they only publish windows and linux images?

Unless you were referring to architecture like AMD64 vs ARM64? That shouldn't be an issue, the directive should be using the inferred platform for selecting the image to copy from. Equivalent to FROM oryd/hydra or docker pull oryd/hydra without --platform= specified for either to change the default.

The example was focused on showing how much simpler the image is without catering to a non-root user, allowing the deployment to explicitly opt-in to non-root by choosing their own UID/GID, instead of needing to be mindful of what was chosen internally.

That's possible since the image doesn't have any real dependency lock-in on a non-root user being configured. They could just as easily take the current non-root images and run them as root in the same manner.